Skip to content

improve CSCL QA app page#2339

Draft
damonmcc wants to merge 4 commits intomainfrom
dm-qa-app-cscl
Draft

improve CSCL QA app page#2339
damonmcc wants to merge 4 commits intomainfrom
dm-qa-app-cscl

Conversation

@damonmcc
Copy link
Copy Markdown
Member

@damonmcc damonmcc commented Apr 30, 2026

related to #2315

improvements to CSCL diff generation and the QA app CSCL page

poc_validation/ scripts

  • run_validation.py (new) — generates diffs by streaming dev and prod files directly from S3, no disk usage for inputs, no full build required
  • validate_outputs.sh — writes validation_summary.csv with per-file prod row counts; mkdir -p so it's re-runnable; adds a note that run_validation.py is now the primary method
  • summarize_diffs.py — reads validation_summary.csv to add # of rows in prod and % of rows with diffs columns to the summary CSV

QA app

  • diff row viewer: select a file with diffs, see the dev-only rows streamed from S3 on demand
  • field-level diffs section: reads from the existing dbt etl_dev_qa models (LION DAT, ThinLION, RPL) grouped by output type
Screenshot 2026-05-03 at 9 55 29 PM Screenshot 2026-05-03 at 9 55 44 PM

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.99%. Comparing base (45fad47) to head (482d42f).

Additional details and impacted files

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@damonmcc damonmcc force-pushed the dm-qa-app-cscl branch 2 times, most recently from 5b464a5 to e53ec4c Compare May 4, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant